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AFIT/ISE/ENV/12-J03 


Abstract 

Air  Force  senior  leadership  has  continued  to  emphasize  the  need  for  progress  on 
improving  cost  and  schedule  estimates  for  military  acquisition  programs.  The  Enterprise 
Requirements  and  Acquisition  Model  (ERAM)  as  modified  by  Leach  and  Searle  is  a  quantitative 
discrete-event  process  simulation  model  accounting  for  activities  from  early  capability  analysis 
through  system  fielding  of  ACAT  I  space  acquisition  programs.  This  research  seeks  to  modify  the 
existing  model  by  including  ACAT  ll/lll  programs  along  with  modeling  the  Rapid  Acquisition 
process  for  space  programs  using  the  same  methodology  and  techniques.  The  model  begins 
with  identification  of  a  desired  space  capability  early  in  the  Joint  Capabilities  and  Integration 
Development  System  (JCIDS)  process  through  system  development  at  Milestone-C  (MS-C)  of  the 
acquisition  system  resulting  in  a  probabilistic  schedule  distribution  for  a  given  concept.  The 
research  focused  on  identifying  activities  and  assigning  duration  distributions  and  probabilities 
based  upon  past  programs  at  each  decision  point.  Data  was  collected  through  analysis  of 
applicable  policy,  instructions,  and  journal  articles  as  well  as  interviews  with  subject  matter 
experts  (SME)  from  the  Air  Staff,  AFSPC  and  the  Space  and  Missile  Systems  Center  (SMC).  ERAM 
has  been  utilized  at  SMC's  space  Concept  Design  Center  (CDC)  providing  program  managers 
insight  into  program  duration  estimations  and  probabilities  of  program  success  for  concepts 
based  on  historical  comparisons.  Defense  Acquisition  University  has  also  found  interest  in  ERAM 
to  be  used  as  a  training  tool  for  personnel  to  better  understand  required  procedures  needed  to 
acquire  a  space  capability  on  schedule  and  within  budget. 
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I.  Introduction 


The  United  States  military  has  enjoyed  unprecedented  strength  in  dealing  with 
its  nations  enemies  over  the  last  thirty  years.  After  the  Vietnam  War,  the  United  States 
rebuilt  its  forces  to  be  unmatched  by  any  country  on  the  planet.  Many  even  argue  that 
this  military  strength  significantly  contributed  to  the  end  of  the  Cold  War,  without  any 
major  conflict  actually  taking  place  (Stein,  1994).  Following  the  end  of  the  Cold  War,  the 
military  was  drawn  down  significantly  but  also  benefitted  immensely  by  upgrades  in 
technology  that  made  it  more  lethal  with  a  smaller  force.  Computer  technology,  GPS, 
and  stealth  are  just  a  few  examples  that  allowed  U.S.  forces  to  soundly  defeat  Iraq  in 
1991  and  then  invade  and  occupy  Afghanistan  and  Iraq  in  2001  and  2003.  The 
Afghanistan  and  second  Iraq  conflicts  did  not  go  as  smoothly  as  the  first  Iraq  war. 
Instead,  both  dragged  on  for  many  years,  forcing  the  military  to  reshape  itself  through 
the  buildup  of  personnel  and  new  equipment. 

The  United  States  pulled  the  last  of  its  forces  out  of  Iraq  in  2011  and  signed  an 
agreement  with  Afghanistan  in  2012  to  have  U.S  forces  leave  by  2014  (Fox  News,  2012). 
Along  with  these  wars,  the  U.S.  economy  has  been  sputtering  since  2008  causing  the 
U.S.  government  to  spend  large  amounts  of  money  to  prop  up  businesses  and,  in  some 
cases,  entire  industries.  These  large  expenses,  along  with  large  recurring  benefit 
expenditures  on  programs  such  as  Medicare,  have  forced  the  U.S.  government  to  relook 
at  spending  across  the  board.  Drawing  down  defense  spending  is  not  abnormal 
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following  conflicts.  Defense  spending  has  decreased  over  29%  each  time  a  conflict  has 


ended  since  the  Korean  War. 


cncr>cr>cncncr>cncr>cncncncncncr>cncncricnooooooo 

H  H  H  t— I  H  H  t— I  H  H  I  rH  rH  H  t— I  H  rH  H  *— I  fN  fN|  CNl  fN  fN  fN  Csl 

Figure  1:  Defense  Spending 

(Office  of  the  Under  Secretary  of  Defense/CFO,  February  2011) 


"I've  proposed  a  new  defense  strategy  that  ensures  we  maintain  the  finest 
military  in  the  world,  while  saving  nearly  half  a  trillion  dollars  in  our  budget"  (Obama, 
2012).  President  Obama's  statement  in  early  2012  was  meant  to  show  that  the  times  of 
large  military  budgets  are  over.  While  politicians  battle  over  this  reduction,  every  one  of 
the  26  people  interviewed  for  this  research  recognize  that  that  a  budget  cut  is  a 
foregone  conclusion.  The  only  thing  they  don't  know  is  how  large  it  will  be.  The  DoD 
has  submitted  a  Fiscal  Year(FY)  2013  budget  for  approval  that  cuts  $31. 8B  from  FY  2012 
spending  (Office  of  the  Under  Secretary  of  Defense/CFO,  2012). 

The  DoD  has  a  very  unique  mission,  unlike  any  other  organization  in  the  world.  It 
is  charged  with  deterring  war,  protecting  the  interests  of  the  United  States  and 


11 


defeating  aggression  anywhere  in  the  world.  Since  it  is  charged  with  such  a  unique 
mission,  the  equipment  that  is  required  is  usually  very  unique  as  well.  In  order  to 
develop  and  field  this  equipment,  the  DoD  has  set  up  acquisition  organizations  that 
focus  on  meeting  the  needs  of  the  warfighters  while  complying  with  a  multitude  of  rules 
and  regulations  prescribed  by  lawmakers  and  DoD  policy  makers.  These  laws  and 
policies  are  the  result  of  the  modern  day  lessons  learned,  process  evolution  and 
lambasting  of  the  acquisition  process  by  a  multitude  of  sources.  When  an  evaluation  of 
the  acqusition  process  starts  off  with  "DOD  is  facing  a  cascading  number  of  problems  in 
managing  its  acquisitions",  major  changes  need  to  take  place  (United  Stated 
Government  Accountability  Office,  2005). 

In  order  to  reverse  the  negative  direction  that  acquisition  programs  had  been 
heading,  the  U.S.  government  enacted  numerous  new  regulations  on  top  of  the  ones 
that  already  existed.  The  most  far-reaching  was  the  Weapons  Systems  Acquisition 
Reform  Act  (WSARA)  of  2009  which  attempted  to  counter  "$295  billion  in  waste  and 
cost  overruns  in  defense  contracts"  (Alexander,  2009).  Despite  these  new  rules  and 
regulations,  in  2012  many  defense  acquisition  programs  are  still  over  budget  and  behind 
schedule.  Continual  work  to  improve  the  process  goes  on  from  within  the  system  as 
program  managers  attempt  to  deliver  the  capability  requested  by  warfighters  while 
staying  within  the  bounds  of  the  budget  and  schedule  they  were  given. 

"We're  running  out  of  money  so  we  must  begin  to  think"  has  been  attributed  to 
Ernest  Rutherford,  the  famous  scientist,  when  he  was  asked  about  the  looming  budget 
crisis  in  his  native  New  Zealand  in  1927  (Luce,  2012).  Director  of  National  Intelligence 
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James  Clapper  has  recently  started  reiterating  that  line  in  speeches  around  Washington 


D.C.  as  he  prepares  the  intelligence  community  for  the  looming  budget  cuts  (Kelly, 
2012).  He  recognizes  that  managers  must  work  smarter  to  achieve  their  goals  in  this 
new  enviornment  of  government  spending  austerity.  As  they  attempt  to  achieve  more 
with  less,  tools  that  facilitate  their  success  are  necessary. 

This  research  augments  a  tool  that  is  available  to  managers  in  space  acquistion 
programs.  Space  programs  have  notoriously  been  over  budget  and  schedule  and 
continue  to  follow  that  precedent.  The  GAO  chart  below  shows  that  the  schedules  for 
seven  major  space  programs  nearly  doubled  from  their  estimates(United  States 
Government  Accountability  Office,  2012).  While  it  is  unclear  if  this  is  because  of  poor 
estimates  or  due  to  programmatic  problems,  it  is  obvious  that  a  better  understanding  of 
space  program  schedules  is  needed. 


Months 

200 


175 


SBIRS  High  GPS  IIF  WGS  AEHF  NPOESS  MUOS  GPS  III 

1996  2000  2000  2001  2002  2004  2008 


Program  and  start  date 


Initial  estimated  number  of  months 
I  Difference  in  estimates 


Source:  GAO  analysis  of  DOD  data. 


Legend:  SBIRS  =  Space  Based  Infrared  System  High;  GPS  =  Global  Positioning  System;  WGS  = 
Wideband  Global  SATCOM;  AEHF  =  Advanced  Extremely  High  Frequency;  NPOESS  =  National 
Polar-orbitinq  Operational  Environmental  Satellite  System;  MUOS  =  Mobile  User  Objective  System. 


Figure  2:  Increase  in  Space  Program  Schedules 
(United  States  Government  Accountability  Office,  2011) 
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Enterprise  Requirements  and  Acquisition  Model  (ERAM) 

A  doctoral  dissertation,  "Identifying  Enterprise  Leverage  Points  in  Defense 

Acquisition  Program  Performance",  was  published  in  September  2009  aiming  to 
characterize  the  system  of  acquiring  large,  complex,  socio-technological  systems  for  the 
DoD  (Wirthlin,  2009).  It  was  an  in-depth  analysis  of  the  discrete  events  and  products 
required  for  a  typical  Air  Force  acquisition  program  throughout  the  lifecycle,  with 
emphasis  placed  on  events  prior  to  Milestone-C  (MS-C).  Data  about  the  events  and 
products  was  gathered  through  analysis  of  existing  policy  and  guidance  as  well  as  many 
interviews  with  defense  acquisition  professionals.  The  data  collected  was  modeled  and 
programmed  into  the  Arena  software  modeling  tool.  This  resulted  in  the  first-ever 
discrete-event  simulation  of  the  entire  defense  acquisition  model,  although  abstracted 
at  an  elevated  level. 

In  the  Fall  of  2010,  the  SMC,  Development  Planning  branch  (SMC/XR)  sponsored 
further  research  and  analysis  on  Wirthlin's  work,  focusing  specifically  on  early  space 
requirements  and  acquisition  activities  for  ACAT  I  programs.  This  resulted  in  a  thesis 
focusing  on  bringing  the  model  up  to  date,  transitioning  to  a  new  software  program  and 
enhancing  it  to  specifically  focus  on  ACAT  I  space  programs.  Leach  and  Searle's  work 
implemented  Wirthlin's  model  and  methodology  while  transitioning  to  the  Extendsim 
software. 

Wirthlin  attempted  to  model  a  complicated  set  of  processes  that  is  run  by  a 
virtual  army  for  the  DoD.  Leach  and  Searle  designated  Wirthlin's  work  as  ERAM  1.0.  As 
this  research  is  an  expansion  of  their  work,  this  research  will  continue  with  this 
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nomenclature.  ERAM  1.0  simulated  various  activities  and  events  using  probabilities  for 


decision  event  outcomes  and  timeline  distributions  to  determine  likely  overall  program 
timelines  as  well  as  probabilities  of  successful  program  execution  up  to  MS-C.  The 
output  is  reflected  by  a  probabilistic  determination  of  likely  program  duration  through 
approval  of  MS-C  using  Monte  Carlo  simulation  techniques.  ERAM  1.0  modeled  a 
capability  concept  or  idea  entering  the  Major  Command  (MAJCOM)  requirements 
process  and  simulated  its  path  to  termination.  Additional  implementation  included 
decision  points  throughout  the  process  flow  to  include  determination  of  a  successful 
design  review,  rework,  funding  checks  and  other  events.  The  activities  for  discrete 
events  incorporated  triangular  distributions  for  elapsed  times.  For  example,  timelines 
associated  with  affordability  assessments,  preparing  for  reviews,  writing  documents,  all 
had  probability  distributions  with  a  best  case,  worst  case  and  most  likely  number  of 
days,  with  the  data  elicited  from  various  sources.  These  sequences  of  events  with 
timeline  probabilities  and  decision  points  as  executed  through  the  simulation  delivered 
results  enabling  further  analysis  for  decision-making.  Accordingly,  with  additional 
refinement  and  enhancements  this  model  has  the  potential  to  be  used  as  a  valuable 
data  source  for  decision  makers  in  forecasting  a  program's  development  and  delivery 
schedules  and  life  cycle  costs  (Wirthlin,  2009). 

Leach  and  Searle  designated  their  work  ERAM  2.1  since  it  was  a  major 
modification  to  Wirthlin's  work.  They  completed  ERAM  2.1  in  June  2011  and  it  was 
subsequently  accepted  by  SMC/XR  for  use  in  the  SMC  Concept  Design  Center  (CDC). 

The  CDC  is  located  in  the  basement  of  the  Charles  C.  Lauritsen  Library  at  the 
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headquarters  of  The  Aerospace  Corporation  in  El  Segundo,  CA.  The  Aerospace 
Corporation  is  a  federally  funded  research  and  development  center  (FFRDC).  1  As  the 
FFRDC  for  national-security  space.  Aerospace  supports  long-term  planning  and  the 
immediate  needs  of  our  nation's  military  and  reconnaissance  space  programs 
(Aerospace  Corp). 

The  CDC  is  a  facility  that  enables  a  cross-functional  design  and  engineering  team 
to  work  in  an  integrated  manner.  The  facility  consists  of  banks  of  networked  computers, 
large  displays  all  built  around  a  centralized  workspace.  It  can  be  seen  in  Figure  3.  The 
facility  is  used  when  a  requirement  for  a  new  space  system  wants  to  be  explored  by  a 
SMC  customer.  A  team  of  subject  matter  experts  (SME)  in  spacecraft  disciplines  such  as 
structures,  propulsion  and  sensors  is  assembled  from  the  Aerospace  staff  to  staff  the 
CDC.  In  addition,  SMEs  in  systems  engineering,  budgeting,  and  acquistion  are  brought  in 
as  part  of  the  team.  This  team  assembles  in  the  CDC  with  customer  representatives  to 
create  a  conceptual  design  including  a  budget  and  schedule  over  the  course  of  2-3  days. 
This  work  allows  decision  makers  to  understand  the  feasibility  of  satisfying  a 
requirment  as  well  as  providing  a  rough  order  of  magnitude  estimate  for  completing  the 
program. 


1  FFRDCs  are  unique  independent  nonprofit  entities  sponsored  and  funded  by  the  U.S. 
government  to  meet  specific  long-term  technical  needs  that  cannot  be  met  by  any  other 
single  organization.  The  Aerospace  Corporation  is  sponsored  by  the  United  States  Air 
Force,  and  provides  objective  technical  analyses  and  assessments  for  space  programs 
that  serve  the  national  interest. 
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Figure  3:  Concept  Design  Center 

SMC/XR  sponsored  the  research  on  ERAM  to  be  used  as  the  tool  for  the 
Acquistion  SMEs  in  the  CDC.  Prior  to  this  work,  the  Acquistion  SME  was  the  only 
member  of  the  team  that  did  not  have  a  tool  and  instead  was  basing  their  work  solely 
on  their  personal  experience.  While  the  Aerospace  SMEs  are  considered  some  of  the 
preeminent  experts  in  their  fields,  SMC  recognized  that  ERAM  could  be  expanded  and 
modified  to  provide  even  better  acquistion  program  estimates  for  their  customers.  The 
acquistion  portfolio  of  SMC  is  actually  currently  comprised  of  50%  ACAT  I  programs  and 
50%  ACAT  ll/lll  and  other  programs. 

In  the  fall  of  2011,  SMC/XR  once  again  sponsored  further  research  and  analysis 
on  ERAM  2.1,  focusing  specifically  on  expanding  the  model  to  include  other  types  of 
acquisition  programs.  They  specifically  wanted  to  diversify  the  model  so  that  it  would 
include  ACAT  II,  ACAT  III  and  rapid  acquisition  programs.  These  programs  have  different 
rules  and,  in  some  cases,  processes  that  govern  how  they  are  managed  and  executed. 
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The  goals  of  SMC/XR  to  update  and  expand  the  work  of  Leach  and  Searle  are  covered  by 


the  three  research  objectives  listed  below. 

1.  Review  and  update  ERAM  2.1  to  ERAM  2.2  for  space  ACAT  ll/lll  programs. 

2.  Review  and  update  ERAM  2.1  to  ERAM  2.2  for  space  rapid  acquisition  programs. 

3.  Identify  additional  research  requirements  for  future  versions  of  ERAM 
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II.  Literature  Review  and  Background  Information 


This  topic  required  the  review  of  more  than  60  policy  documents,  official 
instructions,  guidance,  journal  articles,  government  reports  and  briefings.  As  this 
research  follows  closely  on  the  heels  of  the  Leach  and  Searle  2011  paper  "Department 
of  Defense  Enterprise  Requirements  and  Acquisition  Model,"  it  served  as  the  primary 
baseline  for  this  paper.  Their  research  was  the  first  attempt  at  modeling  the  space 
acquisition  system.  All  studies  prior  to  their  work  such  as  GAO-11-590T  and  GAO  08- 
552T  point  out  the  problems  in  the  space  acquisition  process  and  offer  solutions  but 
don't  attempt  to  model  it. 

Due  to  this  fact,  the  core  documents  that  were  reviewed  to  expand  ERAM  were 
the  same  as  the  ones  used  by  Leach  and  Searle.  More  detail  on  them  can  be  found  in 
Leach  and  Searle's  work,  listed  in  Appendix  III.  Table  2  represents  the  documents  that 
played  the  greatest  importance  to  this  effort. 


Table  1:  Primary  Documents 


Resource 

1) 

Leach,  David  A.  and  Searle,  Chad  T.,  Department  of  Defense  Enterprise 
Requirements  and  Acquisition  Model 

2) 

Strategic  Command  Instruction  534-19  Operationally  Responsive  Space 

3) 

Department  of  Defense  Instruction  5000.02  Operation  of  the  Defense 
Acquisition  System 

4) 

Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170. 01H  Joint  Capabilities 
Integration  and  Development  System 
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5) 

Air  Force  Instruction  63-114  Quick  Reaction  Capability  Process 

6) 

Title  10,  United  States  Code  Armed  Forces 

1.  Leach,  David  A.  and  Searle,  Chad  T.,  Department  of  Defense  Enterprise 
Requirements  and  Acquisition  Model 

Leach  and  Searle's  work  served  as  the  beginning  point  for  this  research,  as  it  is 
an  expansion  of  the  model  that  they  built  in  collaboration  with  SMC/Aerospace  Corp. 
Their  work  begins  with  the  identification  of  a  desired  space  capability  early  in  the  JCIDS 
process  through  system  development  at  Milestone-C  of  the  acquisition  system  resulting 
in  a  probabilistic  schedule  distribution  for  a  given  concept.  Their  version  of  ERAM 
provided  a  comprehensive  early  schedule  estimate  based  upon  existing  government 
policy  and  instructions. 

They  explored  all  available  space  acquisition  policy,  regulations,  statutes,  existing 
practices  and  expertise  from  SMEs  to  convert  Wirthlin's  original  Air  Force-wide  ERAM 
model  into  a  space  acquisition  specific  model  that  could  be  applied  to  SMC's  early 
systems  engineering  efforts  at  the  Concept  Design  Center.  In  addition,  through  their 
research  they  were  able  to  identify  key  challenges  to  space  capability  development  and 
propose  solution  to  each. 
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Table  2:  Key  Challenges  to  Space  Acquisition 


Challenge 

Solution 

Inadequate  early  systems  engineering 

Implement  early  systems  engineering  prior  to  MDD 
and  MS-A 

Failure  of  the  government  to  assess 
technology  maturity 

Improve  technical  knowledge  base  and  establish 
improved  evaluation  methods 

Improper  distribution  of  personnel  between 

HQ  and  program  offices 

Reallocate  personnel  to  program  offices  and  training 
centers  for  them 

Lack  of  process  discipline 

Centralize  processes  to  ensure  processes  are 
standardized 

Poor  6X  Training  and  evaluation  program 

AF  expand  training  for  acquisition  professionals 
beyond  DAU  requirements 

Process  disconnects  between  MAJCOM  and 

AFRL 

Technology  requirements  and  availability  need  to  be 
better  passed  back  and  forth 

2.  Strategic  Command  Instruction  534-19  Operationally  Responsive  Space 

While  no  one  document  establishes  the  operationally  responsive  space  process 
(ORS),  this  instruction  provides  the  best  information.  It  establishes  the  responsibilities 
of  the  organizations  within  the  process,  explains  the  steps  within  the  process  and  also 
lays  out  the  goals  of  the  ORS  program.  The  interesting  point  about  this  instruction  is 
that  it  is  written  from  the  perspective  of  USSTRATCOM,  which  never  has  been  in  the 
chain  of  command  of  the  ORS  Office.  Instead,  their  main  role  is  to  supply  requirements 
for  consideration  to  the  DoD  Executive  Agent  for  Space.  Therefore,  while  this 
instruction  seems  directive  to  all  parties,  it  appears  the  authors  took  liberties  with  their 
authority  and  therefore  it  needs  to  taken  with  a  grain  of  salt. 
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3. 


Department  of  Defense  Instruction  5000.02  Operation  of  the  Defense  Acquisition 


System 

DoD  5000.02  details  the  procedures  to  operate  the  acquisition  system.  It  has 
been  modified  numerous  times  with  policy  letters  and  addendums  but  still  serves  as  the 
baseline  document  for  acquisition  officials.  For  this  research,  it  was  especially 
important  because  it  details  the  difference  between  ACAT  levels  and  the  processes  that 
must  be  followed  for  each. 

4.  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3170. 01H  Joint  Capabilities 
Integration  and  Development  System 

CJCSI  3170. 01H  was  published  in  early  2012  to  bring  together  various  documents 
concerning  the  JCIDS  process.  For  this  research,  it  was  important  because  it  details  the 
requirements  for  a  request  to  be  approved  as  an  urgent  operational  need  and  the  joint 
process  for  fulfilling  urgent  operational  needs  with  rapid  acquisition  programs.  It  also 
specifies  what  steps  may  be  skipped  in  the  JCIDS  process  to  help  expedite  rapid 
acquisitions. 

5.  Air  Force  Instruction  63-114  Quick  Reaction  Capability  Process 

AFI  63-114  is  the  authoritative  document  on  how  Air  Force  acquisition  units  are 
supposed  to  respond  to  urgent  operational  needs.  It  specifies  the  steps  that  were  used 
as  the  starting  point  for  modeling  the  Air  Force  rapid  space  acquisition  process  in  ERAM 
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2.2.  While  it  is  supposed  to  be  authoritative,  not  all  Air  Force  units  utilize  this  document 


but  instead  have  authority  to  operate  using  other  processes. 

6.  Title  10,  United  States  Code  Armed  Forces 

Title  10,  United  States  Code  is  the  listing  of  federal  statutes  that  specify  the 
operation  of  the  United  States  Armed  Forces.  It  lays  out  the  legal  requirements  that 
DoD  and  service  regulations  must  comply  with  when  operating  the  military  acquisition 
system.  It  was  important  to  this  research  because  it  specifies  approval  authorities, 
reporting  requirements  and  cost  assessment  responsibilities  for  programs.  This  affected 
the  ERAM  2.2  model  because  some  requirements  are  different  for  certain  types  of 
programs. 

Acquisition  Categories 

Acquisition  programs  are  categorized  based  on  their  overall  cost  and  interest  by 
law/policy  makers  and  acquisition  approval  authorities  (USD(AT&L),  2008).  Table  1  lays 
out  the  basic  criteria  for  each  category  as  specified  in  DoDI  5000.02.  In  most  cases,  the 
total  cost  of  the  program  drives  it  to  a  certain  category.  Rapid  programs  can  fit  into  any 
of  the  categories  but  normally  are  ACAT  III  or  not  Programs  of  Record.  In  addition  to  the 
three  categories,  there  are  many  acquisition  programs  that  never  become  a  Program  of 
Record.  A  Program  of  Record  is  recorded  in  the  Future  Years  Defense  Program(FYDP), 
essentially  the  listing  and  budgeting  of  all  approved  acquisition  programs.  If  an 
acquisition  program  is  not  a  Program  of  Record,  then  it  is  funded  from  the  services 
operational  budget  and  therefore  not  given  an  acquisition  category. 
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Table  3:  Acquisition  Categories 


ACAT  1 

•  Designated  by  the  USD(AT&L)  as  Major  Defense  Acquistion  Program 

•  Dollarvalue:  estimated  by  the  USD(AT&L)  to  require  an  eventual  total 
expenditure  for  research,  development,  test  and  evaluation  (RDT&E)  of  more  than 
$365  million  in  fiscal  year  (FY)  2000  constant  dollars  or,  for  procurement,  of 

more  than  $2,190  billion  in  FY  2000 constant  dollars 

•  MDA  designation  as  special  interest 

ACAT  II 

•  Does  not  meet  criteria  for  ACAT  1 

•  Dollar  value:  estimated  by  the  DoD  Component  Head  to  require  an 
eventual  total  expenditure  for  RDT&E  of  more  than  $140  million  in  FY  2000 
constant  dollars,  or  for  procurement  of  more  than  $660  million  in  FY  2000 
constant  dollars 

•  USD(AT&L)  designation 

ACAT  III 

Does  not  meet  the  criteria  for  ACAT  II  or  above 

Non-Program 

of  Record 

•  Program  not  listed  in  FYDP 

DoDI  5000.02  lays  out  the  baseline  acquisition  processes  that  must  be  followed 
for  the  different  different  ACAT  levels.  ACAT  I  programs  have  the  most  stringent 
process  while  ACAT  III  have  the  least  stringent.  These  processes  are  laid  out  as  the 
minimum  actions  that  must  be  taken  for  each  program  but  they  do  not  specify  the 
details.  Each  service,  and  in  most  most  cases  the  individual  product  center,  details  the 
specific  steps  that  must  be  followed  to  be  in  compliance  with  the  statutory,  regulatory 
and  policy  laws  and  regulations. 

While  this  is  never  overtly  stated  in  DoDI  5000.02,  acquistion  classification  and 
processes  are  essentially  based  on  the  level  of  risk  of  the  program.  This  is  obvious  in  the 
criteria  for  classification,  the  cost  of  the  program  or  risk  assessment  by  DoD  leaders.  In 
an  effort  to  avoid  risks,  policy  makers  have  chosen  to  implement  more  stringent 
controls  for  the  more  risky  programs.  A  good  example  of  this  is  the  Milestone  Decision 
Authority  for  ACAT  I  programs  is  the  Undersecretary  of  Defense  for  Acquisition, 
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Technology  and  Logistics.  Normally  a  service  designee  (usually  a  3-star  general)  is  the 


Milestone  Decision  Authority  for  ACAT  II//III  programs.  As  one  program  manager 
succinctly  stated  in  an  interview,  "large  programs  are  essentially  more  risky  and 
therefore  too  big  to  fail.  They  get  lots  of  oversight  because  either  the  benefit  is  so  great 
or  the  cost  is  too  big  to  swallow." 

ERAM  2.0  application  and  validation 

In  February/March  2012,  SMC/XR  collaborated  with  former  employees  of  the 

Transformational  Satellite  Communications  System  (TSAT)  program  to  evaluate  the 
ability  of  ERAM  to  predict  schedules.  The  TSAT  program  was  a  new  satellite  program 
intended  to  yield  huge  increases  in  secure  communications  bandwidth  to  U.S  military 
forces.  It  was  cancelled  in  2009  in  an  effort  to  reduce  defense  spending  (Brinton, 

2009). 

SMC/XR  evaluated  ERAM  in  a  three  step  process.  First,  they  ran  the  program 
themselves  based  on  data  they  were  able  to  gather.  They  then  consulted  with  two 
former  members  of  the  program  office  to  gain  a  better  understanding  of  the  program. 
After  the  consultation,  they  changed  their  inputs  in  ERAM  based  on  the  new  data  they 
acquired.  During  the  first  run  with  no  input  from  the  former  TSAT  program  employees, 
ERAM  estimated  that  the  time  to  pass  Milestone  B  would  have  a  mean  of  approximately 
150  months  .  The  actual  program  was  scheduled  to  pass  Milestone  B  at  130  months. 
(Figure  4)  Since  they  had  only  some  information  on  the  program,  they  utilized  the 
Decision  Cube,  which  varies  parameters  of  the  simulation  such  as  technology  readiness. 
For  more  information  on  the  Decision  Cube,  see  Leach  and  Searle's  work. 
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Histogram  of  Time  to  Pass  MS  B  Decision  (Months) 

(for  Sole  Source,  this  is  time  to  enter  Pre-MS  C) 


0.18 

■  Worst  Case  Decision  Cube 


Figure  4:  Time  to  Pass  MS-B  (No  SME  input) 

Following  their  consultation  with  the  former  TSAT  program  employees,  SMC/XR 

then  reran  ERAM  based  on  the  new  input  from  the  TSAT  program.  This  resulted  in  a 

much  better  approximation  for  the  amount  of  time  to  pass  Milestone  B.  (Figure  5) 

ERAM  predicted  a  mean  of  approxamitely  128  months  which  was  signifigantly  closer  to 

the  real  TSAT  schedule. 

While  this  work  lends  some  credibility  to  ERAM,  one  trial  is  not  enough  to 
validate  the  software.  SMC/XR  would  like  to  continue  to  work  with  other  programs  to 
continue  the  validation  but  has  had  little  success  in  getting  cooperation.  In  general, 
program  managers  politely  decline  because  they  need  their  time  to  focus  on  their 
current  program.  Most  likely,  ERAM  validation  will  take  place  as  new  acquistion 
programs  pass  through  the  CDC  and  then  their  actual  results  are  compared  to  the 
predictions  made  by  ERAM  at  their  initiation. 
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Histogram  of  Time  to  Pass  MS  B  Decision  (Months) 

(for  Sole  Source,  thisis  time  to  enter  Pre-MS  C) 


Figure  5:  Time  to  Pass  MS-B  (With  SME  input) 

Despite  the  fact  that  this  evaluation  of  ERAM  only  looked  at  one  program, 

SMC/XR  and  Aerospace  were  able  to  draw  some  conclusions  about  the  application  of 

ERAM.  They  called  this  group  of  conclusions  "Every  Program  has  a  Story"  because  it 

became  obvious,  after  looking  at  TSAT,  that  each  acquistion  program  is  unique.  The 

bottom  line  that  they  learned  from  this  evaluation  was  that  ERAM  needs  to  be  more 

flexible  in  describing  a  "program's  story"  (Broder,  2012). 

1 .  “Real  World’’  factors  (outside  the  Air  Force)  had  a  great  deal  of  impact 
on  determining  the  programs  start  and  demise 

2.  TSAT  (and  many  programs)  can  be  very  dynamic  so  that  model  inputs 
that  are  accurate  at  the  start  can  change  year  over  year 

3.  The  acquisition  process,  as  depicted  on  the  DAU  Wall  Chart,  does  not 
represent  TSAT’s  (or  many  program ’s)  actual  path 
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4.  Oral  histories,  as  much  as  any  collection  of  briefing  charts,  help  describe 
the  important  underlying  issues  in  acquisition 

The  conclusions  that  SMC/XR  drew  support  the  motivation  that  has  driven  ERAM 
from  the  very  beginning.  The  acquisition  world  is  complex  and  not  always 
representative  of  the  documented  processes.  This  research  strived  to  expand  and 
modify  the  model  so  that  it  is  more  representative  of  the  "real  world". 
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III.  Methodology 

This  research  was  conducted  utilizing  qualitative  social  science  and  quantitative 
evaluation  with  the  goal  of  modifying  and  expanding  an  existing  quantitative  discrete 
event  simulation  model  of  space  acquisition  program  timelines  to  be  utilized  by  SMC  as 
a  decision  support  tool  during  early  concept  analysis.  As  this  is  an  expansion  of  the 
work  conducted  by  Leach  and  Searle,  it  utilizes  the  same  methodology  as  their  research 
but  focused  on  different  organizations  and  data  sets. 

Research  Scope 

The  focus  of  this  research  is  to  analyze  and  model  the  discrete  events  for  Air 
Force  ACAT  ll/lll  space  programs  from  capability  gap  analysis  through  MS-C  of  the 
acquisition  system.  Additionally,  rapid  space  acquisition  programs  were  analyzed  and 
modeled.  Since  these  programs  are  run  according  to  significantly  different  processes 
than  normal  acquisition  programs,  this  research  focused  from  requirement  submission 
all  the  way  to  initial  fielding  of  a  capability.  Focusing  on  the  front  end  of  ERAM  limited 
the  literature  reviews  and  key  personnel  interviews  to  OSD,  Acquisition,  Technology  & 
Logistics  (OSD/AT&L),  Secretary  of  the  Air  Force,  Acquisitions  (SAF/AQ),  AFSPC,  SMC,  the 
Aerospace  Corporation  and  government  support  contractors.  Due  to  the  fact  that 
AFSPC  maintains  the  cyber  and  information  technology  (IT)  domain  within  their 
portfolio,  some  of  the  interviews  related  to  these  areas  but  the  modeling  remained 
focused  on  space  acquisition  since  SMC  was  not  interested  in  these  topics. 
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As  in  ERAM  2.0,  the  PPB&E  and  Test  and  Evaluation  (T&E)  processes  were  not 


expanded  beyond  what  was  previously  developed  in  ERAM  1.0.  PPB&E  is  the  calendar 
driven  funding  process;  the  detailed  formal  budgeting  process  will  be  a  topic  left  to 
future  research  to  increase  the  fidelity  of  ERAM.  Certain  elements  in  the  model  have 
incorporated  checks  for  available  funding,  and  above-threshold  cost  increases. 

However,  it  does  not  identify  the  specific  activities  and  decision  points  as  well  as  their 
duration  distributions  and  probabilities,  respectively.  Uncertainty  events  which  occur  in 
the  model  may  have  ties  to  budget  directives  and  decisions  and  will  be  based  on  out-of- 
cycle  budget  cut  drills  and  other  events  gleaned  from  ERAM  1.0 

Also,  analysis  of  the  T&E  activities  identified  in  ERAM  1.0  was  not  included  in  the 
scope  of  this  research  for  ACAT  ll/lll  programs.  Since  rapid  program  obviously  take  place 
very  quickly,  T&E  efforts  were  able  to  be  studied  at  a  high  level  and  this  is  included  in 
the  updated  model.  Since  it  is  such  a  important  and  time-intensive  activity  in  most 
program,  this  is  a  prime  topic  for  additional  research,  e.g.  identify,  document  and 
update  ERAM  with  the  differences  between  aerospace  and  space  ACAT  l/ll/lll  T&E 
activities. 

Research  Objectives 

At  the  beginning  of  the  research  effort,  the  research  objectives  from  Leach  and 
Searle's  work  were  revised  and  adjusted  to  meet  the  new  goals  of  this  iteration.  The 
following  objectives  were  identified  and  agreed  upon  by  both  the  customer  (SMC/XR)  as 
well  as  the  research  team  (AFIT/ENV). 

1.  Review  and  update  ERAM  2.1  to  ERAM  2.2  for  space  ACAT  ll/lll  programs. 
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2.  Review  and  update  ERAM  2.1  to  ERAM  2.2  for  space  rapid  acquisition  programs. 


3.  Identify  additional  research  requirements  for  future  versions  of  ERAM 

Process 

Due  to  the  fact  that  this  research  effort  was  an  expansion  and  modification  of  the  work 
completed  by  Leach  and  Searle,  their  approach  was  analyzed  for  applicability  and 
adopted  with  some  minor  modifications. 

1.  Background  Information  Collection  and  Review  -  Review  previous  research, 
policy,  instructions,  guides 

2.  ERAM  2.1  Analysis  -  Familiarization  with  current  model 

3.  Conduct  Interviews  -  Identify  subject  matter  experts  and  conduct  interviews  on 
experience  relevant  to  ERAM  2.2  research  objectives 

4.  Model  Processes  and  Implement  updates  to  develop  ERAM  2.2  in  Extendsim 

5.  Report  and  Brief  Findings  -  Write  final  report,  brief  findings  and  provide  data  to 
SMC/XR 

Step  1:  Background  Information  Collection  and  Review 

The  first  step  was  to  gather  and  evaluate  the  applicable  DoD  and  Air  Force 
directives  and  publications  that  pertain  to  the  research  topic.  In  addition,  related 
research  on  the  topic  was  collected  and  evaluated  for  inclusion  in  the  model.  Fig  6 
provides  some  context  on  the  variety  and  depth  of  the  policy  and  directives  that  space 
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acquisition  program  managers  must  comply  with.  The  documents  listed  were  the 
primary  sources  for  this  step  in  the  research  process. 


Key  Policy  Memos 
OSD  Memo:  Acquisition  Actionsin  Support 
of  Joint  Urgent  Operational  Needs 
USD/AT&L  DTM  09-025:  Space  Systems 
Acquisition  Policy 

SAF  Memo:  HQ/AFSpace  Management  & 
Organization 


Key  Reference  Documents 
Lt  Col  Wirthlin:  Identifying  Enterprise 
Leverage  Points  in  Defense  Acquisition 
Program  Points 

Leach  and  Searle:  Department  of  Defense 
Enterprise  Requirements  and  Acquisition 
Model 


Figure  6:  Key  Documents  for  ERAM  2.2 
Step  2:  ERAM  2.1  Analysis 

The  next  step  was  to  learn  how  the  space  acquisition  process  was  modeled  in 
ERAM  2.1.  Since  it  modeled  as  a  discrete  event  simulation,  it  was  necessary  to  identify 
the  key  points  in  the  model  that  would  need  to  modified  or  rebuilt.  ERAM  2.1  was 
modeled  in  the  ExtendSim  software  package.  It  utilizes  a  graphic  interface  to  model 
processes.  There  are  numerous  operations  that  can  be  chosen  for  each  icon  and  the 
operations  parameters  are  configurable  based  on  what  that  icon  is  supposed  to 
represent.  Figure  7  is  a  close  up  view  of  the  ERAM  2.1  as  it  was  modeled  in  ExtendSim. 
It  shows  a  small  portion  of  the  model,  which  actually  has  over  1000  icons.  Leach  and 
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Searle  explain  some  of  the  more  common  icons  and  their  operations  in  ExtendSim  very 


well  using  the  example  icons  shown  in  Figure  8  (Leach  &  Searle,  2011). 


"Event/Activity"  icon  is  implemented  with  a  time  duration  allowing  a 
distribution  to  be  selected.  For  this  research,  triangular  distributions  were 
implemented  based  on  the  data  elicited  from  personnel.  The  "And  Merge  With 
Wait"  waits  for  all  the  inputs  to  arrive  before  proceeding  to  the  next  event.  The 
"Or"  icon  uses  Boolean  logic  to  proceed  if  either  of  the  inputs  occur.  The 
"Decision  Point"  output  is  based  on  the  likelihood  of  an  event  to  occur.  The 
probabilities  of  each  event  are  entered  into  the  properties  of  the  icon. 


Figure  7:  Example  ERAM  Icon  Flow 
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ExtendSim  Icon 


Function  Name 


pel)41 

Clinger-Cohen  Assess/Cert 

Event/Activity 

[g]r[2336]-T|J 

And  Merge  with  Wait 

m  Qa 
p 

Bring  3  processes  together  PreB 

1  jaiMIKT, 

Or 

_J4317][391i 

SDR  2 

Decision  Point 

=j!miE= 

Submodel 

Um= 

PDR  submodel 

Figure  8:  ERAM  Icons 


Step  3:  Conduct  Interviews 

The  third  step  was  to  conduct  interviews  with  subject  matter  experts  familiar 
with  capability  gap  analysis,  requirements  development  and  the  acquisition  process. 
These  interviews  were  scheduled  using  purposeful  and  snowball  sampling.  The 
interview  itself  was  conducted  in  a  semi-formal  manner  where  the  questions  were 
decided  ahead  of  time  by  the  researcher  but  the  conversation  was  allowed  to  deviate 
from  the  pre-prepared  questions.  Purposeful  sampling  identified  known  SMEs  (military, 
government  civilians.  Federally  Funded  Research  &  Development  Contract  personnel 
and  support  contractors)  from  SMC,  AFSPC,  SAF/AQ.  and  other  offices  involved  in  space 
acquisition  based  on  their  current  or  past  jobs  in  the  acquisition  and  requirements 
fields.  The  interviews  provided  a  perspective  on  how  and  where  policy  makers  and 
senior  leaders  are  most  likely  to  impact  timelines  and  coordination  cycles  on  required 
acquisition  documentation.  All  the  interviews  concluded  with  a  request  for 
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recommendations  of  additional  SMEs  hence  the  use  of  snowball  sampling.  A 
representative  list  of  questions  used  for  the  interviews  can  be  found  in  Appendix  II. 

The  concept  of  triangulation  was  utilized  to  aid  in  improving  results. 
Triangulation  is  the  concept  of  approaching  the  same  topic  from  multiple  angles 
(Neuman,  2006).  This  was  utilized  by  talking  to  SMEs  at  different  levels  (program  office, 
headquarters,  and  intermediate  headquarters)  about  the  same  acquisition  programs. 

By  talking  to  each  of  them  about  the  same  topic,  research  data  was  more  easily 
verifiable.  Further,  existing  quantitative  data  such  as  documents,  policies,  actual 
documented  process  task  durations  constituted  another  approach  to  verify  data. 

An  interesting  issue  conducting  interviews  was  the  initial  lack  of  willingness  by 
respondents  to  provide  data.  Many  were  willing  to  talk  about  their  experiences  and 
their  opinions  on  "how  the  system  works"  but  did  not  want  to  share  data  on  their 
programs.  When  questioned  for  their  reasoning,  all  responded  that  even  though  the 
data  was  not  classified,  they  did  not  want  any  programs  to  be  criticized.  This  response 
was  most  evident  by  the  people  working  at  headquarters  levels  where  they  had  access 
to  information  on  multiple  programs.  This  concern  was  mitigated  by  explaining  that  all 
data  would  be  amalgamated  into  the  model  with  no  programs  specified.  This  practice 
was  followed  for  all  results  except  for  the  ORS  model.  Data  for  only  one  program  was 
available  for  this  model. 

Step  4:  Model  Processes  and  Implement  updates  to  develop  ERAM  2.2 
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The  next  step  of  the  process  was  to  amalgamate  all  the  documents  and 


interviews  into  the  ERAM  model.  ERAM  2.1  was  designed  with  break  points  to  allow 
modifications  to  be  made  for  ACAT  ll/lll  programs.  In  most  cases,  the  process  is  the 
same  or  very  similar  but  the  parameters  for  some  of  the  steps  are  different.  This 
allowed  the  researcher  to  easily  work  with  the  SMC/XR  and  Aerospace  teams  that  are 
charged  with  modifying  ERAM.  Some  examples  of  changes  were  to  the  Document 
Approval  Paths.  ACAT  I  programs  had  a  distribution  with  much  larger  "Most  Likely"  and 
"Maximum"  values  than  ACAT  ll/lll  programs  so  the  model  had  to  modified  to  show  this 
disparity. 

Rapid  programs  were  not  accounted  for  in  ERAM  2.1.  Similarly,  after  researching 
the  processes  currently  followed  for  rapid  programs,  it  was  obvious  that  they  are  mostly 
exempt  from  the  processes  represented  in  ERAM  2.1.  Due  to  these  facts,  a  new  model 
had  to  be  built  within  ERAM  2.2  to  represent  the  two  rapid  space  acquisition  processes 
that  data  was  available  for. 

Step  5:  Report  and  Brief  Findings 

The  results  of  the  first  four  steps  were  compiled  and  are  presented  in  this 
research  report.  A  brief  synopsis  of  the  main  publications  that  were  reviewed  on  this 
topic  is  presented  along  with  the  notes  from  the  interviews,  information  from  the  policy 
documents,  and  an  understanding  of  the  ERAM  2.2  model  along  with  comments  about 
the  space  acquisition  business.  Finally,  the  results  of  the  research  that  were  used  to 
modify  the  ExtendSim  code  to  create  ERAM  2.2  from  ERAM  2.1  are  presented. 
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IV.  Air  Force  Space  Rapid  Acquisition 

Rapid  acquisition  has  received  a  lot  of  attention  in  recent  years  as  conflicts  in 
Iraq  and  Afghanistan  have  created  a  need  to  quickly  adapt  to  a  group  of  extremely 
cunning  enemies  that  have  utilized  guerrilla  warfare  techniques.  Rapid  acquisition  is 
nothing  new  for  the  military  and  has  been  utilized  it  for  programs  such  as  the  U-2 
aircraft,  the  MC-12  aircraft  and  adding  weapons  to  Predator  unmanned  reconnaissance 
aircraft.  Essentially  the  goal  is  to  work  outside  the  normal  acquisition  process  to  quickly 
field  a  capability  that  is  needed  for  war.  Rapid  acquisition  is  defined  by  the  Department 
of  Defense  as: 


a  streamlined  and  tightly  integrated  iterative  approach,  acting  upon 
validated  urgent  or  emergent  capability  requirements,  to:  conduct 
analysis  and  evaluate  alternatives  and  identify  preferred  solutions; 
develop  and  approve  acquisition  documents;  contract  using  all 
available  statutory  and  regulatory  authorities  and  waivers  and 
deviations  of  such,  appropriate  to  the  situation;  identify  and 
minimize  technical  development,  integration,  and  manufacturing 
risks;  and  rapidly  produce  and  deliver  required  capabilities. 
(Department  of  Defense  Joint  Chiefs  of  Staff,  2012) 

A  more  succinct  way  to  define  the  purpose  rapid  acquisition  was  told  by  Defense 

Secretary  Robert  Gates  to  the  media  when  questioned  about  the  importance  of  the 

rapid  program,  "You  have  to  look  outside  the  normal  bureaucratic  way  of  doing  things 

and  so  does  industry  -  because  lives  are  at  stake"  (Gates,  2007). 

The  majority  of  rapid  acquisition  is  focused  on  supporting  Combatant 
Commands'  efforts  during  contingency  operations.  A  recent  example  that  has  received 
a  lot  of  attention  is  the  Mine  Resistant  Ambush  protected  (MRAP)  vehicles.  MRAPs 
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were  quickly  developed  and  procured  for  U.S  forces  in  response  to  insurgents  using 
roadside  improvised  explosive  devices  (lEDs)  in  Iraq  and  Afghanistan.  In  fact,  this 
became  the  top  acquisition  program  for  2007  with  over  1,500  delivered  for  over  $750M 
and  helped  drop  the  death  rate  from  lEDs  by  88%  (Vanden  Brook,  2008). 

Space  acquisition  programs  do  not  generally  fit  the  profile  of  rapid  programs 
since  development  of  satellites  takes  many  years  and  is  usually  in  response  to  a  strategic 
requirement  or  predicted  tactical  shortfall.  Air  Force  Instruction  63-114  titled  "Quick 
Reaction  Capability  Process"  is  the  primary  Air  Force  guidance  for  implementing  rapid 
acquisitions.  While  it  is  the  primary  guidance,  this  research  uncovered  other 
organizations  that  execute  rapid  space  acquisition  utilizing  other  governing  guidance. 
Even  though  there  are  many  offices  within  the  space  acquisition  community  that  can 
manage  rapid  programs,  many  of  them  are  not  necessarily  in  support  of  contingency 
operations.  Each  performs  a  slightly  different  mission  within  the  community  but  they  all 
operate  with  the  promise  of  delivering  a  capability  faster  than  the  normal  acquisition 
process.  Normal  acquisition  processes  are  known  to  take  quite  a  long  time  or  as  one 
program  manager  stated  during  an  interview  "Acquisition  programs  are  not  set  up  to  be 
fast,  like  Congress  with  lots  of  checks  and  balances."  This  research  discusses  the 
different  organizations  that  are  alternatives  to  the  normal  acquisition  process.  The 
following  sections  are  the  result  of  a  compilation  of  various  data  sources,  including  the 
interviews  analyzed  according  to  the  previously  stated  methods. 
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SMC  Rapid  Reaction  Branch 

The  SMC  Rapid  Reaction  Branch  was  created  after  a  1995  independent  review  of 
counterspace  programs  detailed  a  need  for  rapid  acquisitions.  It  is  currently  located  in 
Colorado  Springs.  Their  mission  is  to  rapidly  respond  to  warfighter  requirements  by 
developing  new  technologies  and  transitioning  them  to  existing  programs.  They  focus 
closely  on  the  space  superiority  mission  area  with  the  goal  of  fielding  capabilities 
between  6  months  and  2  years.  While  much  of  their  work  is  very  sensitive,  their  process 
is  detailed  in  Air  Force  Instruction  63-114  Quick  Reaction  Capability  Process.  This 
process  was  used  as  the  baseline  for  modeling  Air  Force  rapid  space  acqusition  in  ERAM. 
This  process  will  be  detailed  later  in  this  chapter. 

Air  Force  Tactical  Exploitation  of  National  Capabilities  (AF  TENCAP) 

The  AF  TENCAP  office,  along  with  Army  and  Navy  TENCAP,  was  established  in 

1977  with  three  primary  missions:  "exploit  space  systems  for  tactical  applications 
through  rapid  prototyping  projects;  influence  the  design  of  future  space  systems  for 
tactical  applications;  and  educate  warfighters  about  the  capabilities  and  tactical  utility  of 
space  systems"  (Air  Force  Space  Command  Public  Affairs  Office,  2010).  Their  goal  is  to 
act  as  a  bridge  between  the  national  intelligence  and  stategic  space  communities  that 
operate  most  space  systems  and  the  operational  warfighter  communities.  AF  TENCAP 
demonstrates  leading  edge  space  technologies  with  potential  to  enhance  combat 
capabilities  of  units  in  the  field,  then  transitions  these  combat  systems  to  warfighters 
much  more  rapidly  than  traditional  acquisition  processes  (Air  Force  Space  Command 
Public  Affairs  Office,  2010). 
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AF  TENCAP  operates  under  the  premise  of  bringing  prototypes  and 


developmental  solutions  to  the  warfighter  at  the  "speed  of  need".  They  work  under  the 
acqusition  authority  of  the  Program  Executive  Officer  for  Space  (PEO  SPACE),  who  is 
currently  the  SMC  Commander.  Interviews  assed  additional  detail  and  context  to  the 
understanding  of  AF  TENCAP 

AF  TENCAP  works  with  a  rather  small  budget($16M)  by  acquistion  program 
standards  but  partners  with  customers  that  want  to  benefit  from  their  work.  These 
customers  injected  over  $45M  into  AF  TENCAP  programs  in  2011. 

AF  TENCAP  accepts  projects  based  on  broad  operational  requirements  set  by  the 
PEO  SPACE,  but  has  two  distinct  operating  principles  that  differtiate  it  from  other  rapid 
acqusition  organizations.  First,  they  constantly  survey  industry  looking  for  new 
technologies  that  could  reap  rewards  by  applying  them  to  military  problems.  Utilizing 
their  own  budget  or  capitalizing  on  the  budget  of  potential  customers,  they  can  plant 
seed  money  into  these  technologies  with  the  hope  that  it  will  blossom  into  a  game- 
changer.  Along  with  these  investments  into  promising  technology,  they  also  accept 
projects  that  have  a  defined  requirement  from  a  warfighter  and  manage  acquistion 
programs  to  meet  these  requirements.  These  programs  must  fit  within  their  charter. 

Second,  AF  TENCAP  is  allowed  to  manage  their  portfolio  of  projects  internally. 
The  commander  of  AF  TENCAP  has  the  ability  to  direct  investment  into  disciplines  and 
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requirements  that  will  result  in  the  highest  reward  for  their  efforts.  They  accept  project 
based  on  a  basic  set  of  principles2: 

1.  Projects  must  bear  fruit  within  18  months. 

2.  Stay  under  $1M  budget,  but  will  go  higher  with  customer  partnership. 

3.  Maximize  Return  on  Investment  (ROI)  or  Impact  of  Investment  (101)  of 
project  for  the  Department  of  Defense. 

4.  Manage  risk  across  portfolio  to  ensure  that  75%  of  projects  succeed. 

While  they  are  considered  a  rapid  organization,  they  do  not  work  within  the 
confines  of  the  Air  Force  Rapid  Acquistion  regulations  but  instead  are  governed  by  the 
an  Air  Force  Program  Management  Directive  TEN1(1).  While  the  majority  of  the  ERAM 
model  for  rapid  space  acquistion  was  based  upon  the  Air  Force  Rapid  Acquistion 
regulations  ,  the  model  had  to  modified  to  accommodate  the  latitude  that  AF  TENCAP  is 
given  to  accomplish  their  mission.  Specifically,  the  fact  that  they  invest  in  new 
technology  without  a  requirement  was  added  to  the  model. 


Operationally  Responsive  Space(ORS)  Office 

The  ORS  office  was  established  in  May  2007  in  response  to  National  Security 

Presidential  Directive  40  issued  in  January  2005  calling  for  an  "initial  capability  for 
operationally  responsive  access  to  and  use  of  space"  (USSTRATCOM/J84,  2008). 
Essentially,  the  ORS  office  is  charted  to  try  to  deliver  essential  space  capabilities  to 
warfighters  quickly.  Recognizing  that  this  effort  would  be  diffucult  under  the  existing 
statutes  and  regulations.  Congress  specifically  stated  that  "JCIDS  would  not  apply"  in  the 
2007  National  Defense  Authorization  Act  (United  States  Congress,  2006).  Along  with 

2  This  set  is  very  similar  to  the  guiding  principles  that  a  small  business  would  operate  under,  instead  of  a 
centrally  managed  structure  that  the  majority  of  the  AF  acquisition  community  uses. 
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exemption  from  the  JCIDS  process,  the  ORS  office  was  established  with  a  shortened 


chain  of  command.  The  ORS  office  is  headed  by  the  DoD  Executive  Agent  for  Space, 
who  is  currently  the  Deputy  Undersecretary  of  the  Air  Force  for  Space  Programs. 

The  ORS  office  takes  on  projects  with  the  goal  of  binning  them  into  three  types 
of  solutions: 

1.  Tier-1  Employment  -  Capability  operational  within  hours  utilizing  existing 
assets 

2.  Tier-2  Deployment  -  Capability  operational  within  weeks  utilizing  field-ready 
capabilities  in  storage 

3.  Tier-3  Development  -  Capability  operational  within  months  after  design, 
assembly  and  deployment 

The  ORS  Office  has  undertaken  many  research  projects  to  build  the  technology  to 
achieve  their  objectives  but  has  only  been  tasked  to  fulfill  four  requirements.  Three 
were  approved  while  one  was  deemed  too  costly  to  fulfill.  The  first  was  to  meet  a  gap 
in  UHF  satellite  communciations  for  the  Navy.  This  was  satisfied  as  a  Tier-1  program  by 
purchasing  additional  bandwidth  from  commercial  communication  providers.  The 
second  was  to  meet  a  gap  in  space  situational  awareness  which  was  fulfilled  by  a  Tier-3 
development  modifying  exisiting  systems  not  designed  for  those  requirements.  The 
third  program  was  to  meet  a  gap  in  intelligence,  surveillance  and  reconnaissance.  The 
ORS-1  satellite  program  was  developed  as  a  Tier-3  program  to  meet  this  need.  ORS-1 
consists  of  a  modified  U-2  aircraft  camera  mated  to  a  satellite  bus  that  had  been 
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developed  under  another  program.  It  is  currently  operational  in  support  of  US  Central 


Command  forces  in  the  Middle  East  and  Afghanistan. 

Since  the  ORS  office  is  exempt  from  JCIDS,  the  process  that  they  follow  is  unique 
to  them  and  had  to  be  modeled  in  ERAM  seperately  from  the  model  based  upon  Air 


Force  Rapid  Acquistion  regulations.  Combatant  Commands  can  submit  requirements  to 
US  Strategic  Command  (USSTRATCOM),  which  vetts  the  requirement  and  submits  it  to 
the  ORS  Executive  Committee  for  approval.  Once  approved,  the  ORS  office  works 
closely  with  USSTRATCOM  and  the  requesting  Combatant  Command  to  fulfill  the 
requirement. 


Needed  capabilities 
employed  to  produce 
requested  effects  for  JFC 


Figure  9:  High  level  ORS  Process 
(USSTRATCOM/J84,  2008) 

The  ORS  Office  is  currently  being  considered  for  closure  in  the  DoD  FY  13  budget 
based  on  the  idea  that  it  has  completed  its  intial  mission  requirements.  While  this  is  not 
finalized  yet,  the  intent  is  for  the  personnel  currently  in  the  program  to  be  distributed 
throughout  the  current  space  acqusition  community  so  they  can  distribute  their 
knowledge. 
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USAF  Rapid  Capabilites  Office 

The  USAF  Rapid  Capabilities  Office(RCO),  located  in  Washington  D.C.  develops 
and  fields  sensitive  weapons  systems  and  combat  support  equipment  on  accelerated 
timelines  (Air  Force  Public  Affairs,  2012).  The  majority  of  the  work  it  performs  is 
classified  but  it  has  been  revealed  that  they  work  on  programs  such  as  the  X-37B  Orbital 
Test  Vehicle,  an  unmanned,  vertical-takeoff,  horizontal  landing,  spaceplane.  In  order  to 
meet  the  accelerated  timelines  that  is  required  for  certain  programs,  the  RCO  utilizes  a 
streamlined  set  of  processes  that  are  compliant  with  all  statutory  guidance  but  can 
receive  waivers  to  burdensome  processes,  procedures  and  regulations.  Further,  they 
tightly  integrate  with  the  customer  so  that  the  customer  is  pulling  for  the  capability 
while  the  RCO  is  pushing  the  product.  The  RCO  was  chartered  with  a  very  narrow  of 
chain  of  command  for  approval.  In  fact,  their  streamlined  leadership  structure  is  a 
Board  of  Directors  composed  of  the  Undersecretary  of  Defense  for  Acquistion, 
Technology  and  Logistics,  the  Secretary  of  the  Air  Force,  the  Chief  of  Staff  of  the  Air 
Force  and  Assistant  Secretary  of  the  Air  Force  for  Acquistion.  This  is  unlike  any  other  Air 
Force  acqusition  organization  which  normally  has  multiple  layers  of  decision  makers  of 
various  ranks.  The  final  piece  that  makes  the  AF  RCO  different  is  that  their  funding 
sources  are  seen  as  stable.  Due  to  the  the  fact  that  all  the  AF  RCO  work  is  extremely 
sensitive  and/or  classified,  this  research  did  not  attempt  to  include  their  processes  in 
the  ERAM  model. 
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Analysis  of  Rapid  Space  Acquisition 

After  analyzing  the  different  organizations  and  processes  available  for  acquiring 

a  new  capability  rapidly,  it  became  obvious  that  there  were  essentially  three  methods 
available.  AFI  63-114  specifies  the  standard  process  but  AF  TENCAP  and  the  ORS  Office 
have  variations  that  are  available  to  them.  Budgeting  was  not  modeled  for  any  rapid 
programs  because  it  usually  is  the  responsibility  of  the  requestor.  Without  funding 
available,  the  requirement  is  usually  not  even  submitted  for  consideration. 

The  process  specified  in  AFI  63-114  and  the  AF  TENCAP  process  were  combined 
into  the  block  diagram  layed  out  in  Fig  10.  Statistical  distributions  for  the  time  to 
complete  the  steps  in  the  process  were  derived  based  on  past  programs  that  entered 
into  the  process.  Also,  there  are  points  within  the  process  where  decisionmakers 
choose  which  path  the  process  continues  down.  Each  path  at  the  decision  points  was 
assigned  a  probability  based  on  historical  data  gathered  from  interviews  and  documents 
provided  by  subject  matter  experts.  Due  to  the  fact  that  extremely  detailed  information 
was  not  available  and/or  the  complexity  of  representing  the  process  in  ExtendSim,  some 
of  the  steps  are  combined  into  larger  steps  for  the  simulation.  The  combined  steps  are 
represented  by  the  gray  boxes  and  they  are  assigned  a  box  number.  Also,  they  are 
identified  as  either  duration  steps  or  decision  steps.  Some  steps  have  different 
distributions  depending  on  if  the  acquistion  will  be  completed  by  TENCAP  or  by  other 
organizations.  The  corresponding  data  that  will  input  into  ExtendSim  for  these  steps  is 
shown  in  Tables  4,5,  and  6.  The  duration  data  represents  the  number  of  days  that  have 
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passed  since  the  previous  step.  The  probability  data  represents  the  probability  that 
path  will  be  chosen  in  the  model. 


complete 


Figure  10:  Air  Force  Rapid  Space  Acquisition  Process 


Table  4:  QRC  Duration  Data 


Max 

Min 

Most  Likely 

(days) 

(days) 

(days) 

Warfighter  submits  requirement 

Lead  command  assigned  and  requirement 

Box  1 

Validation 

121 

1 

25 

Box  3 

Development 

360 

87 

117 

Box  4 

Assess  Suitability 

62 

11 

36.5 

Box  5 

Adjust  solutions 

273 

10 

141.5 

Box  3  (Duration/ 
Decision) 


Development 
►  (QRC  or  TENCAP 
distribution) 


Initial  Fielding 
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Table  5:  TENCAP  Duration  Data 


Max 

Min 

Most  Likely 

(days) 

(days) 

(days) 

Requirement 

Box  7 

Requirement  approved 

109 

1 

12 

Box  3 

Development 

317 

45 

228 

Box  4 

Assess  Suitability 

62 

2 

12 

Box  5 

Adjust  solutions 

242 

1 
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Table  6:  Rapid  Program  Decision  Probabilities 


Box  1 

Not  validated 

33.00% 

Box  1 

Validated 

66.67% 

Box  2 

Propose  QRC  acquisition 

90.00% 

Box  2 

Non-material  solution 

5.00% 

Box  2 

Address  through  normal  channels 

5.00% 

Box  3 

Continue  to  initial  fielding 

83.33% 

Box  3 

Cancelled  due  to  lack  of  progress 

16.67% 

Box  4 

Evaluation  complete 

90.00% 

Box  4 

Adjust  solutions 

10.00% 

Box  6 

Sustain  existing 

20.00% 

Box  6 

Transition  enduring 

60.00% 

Box  6 

Demil/dispose 

20.00% 

The  ORS  Office  has  a  significantly  different  process  than  other  rapid  acquisition 
organizations.  They  are  exempt  from  JCIDS  and  USSTRATCOM  acts  as  the  gatekeeper 
for  all  requirements.  The  ORS  Executive  Committee  determines  if  a  requirement  should 
be  assigned  to  the  ORS  office  and  also  procures  funding.  The  ORS  process  was  modeled 
separately  in  ERAM  and  is  represented  by  the  block  diagram  in  Fig  11.  The  durations  for 
steps  and  probabilities  of  decisions  are  shown  in  Tables  7  and  8.  Due  to  the  fact  that 
this  process  was  only  exercised  four  times  and  only  produced  one  acquisition  program 
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(ORS-1),  it  is  very  likely  that  the  model  may  not  be  very  accurate  as  a  predictive  tool  for 


future  programs.  As  an  example,  process  step  durations  are  not  represented  by  a 
triangular  distribution  because  there  is  not  enough  data  available  to  build  a  distribution. 


Box  3 
(Duration/ 
Decision) 


Box  4 
(Duration) 


Figure  11:  ORS  Process 


Table  7:  ORS  Duration  Data 


Duration  (days) 

COCOM  submits  requirement 

Box  1 

USSTRATCOM  evaluates  requirement 

61 

Box  2 

Requirement  approved  by  Exec  Committee 

153 

Box  3 

Solution  developed 

243 

Box  4 

Development 

730 

Table  8:  ORS  Decision  Probabilities 


Box  1 

STRATCOM  determines  capability  already  exists 

25% 

Box  1 

Requirement  approved 

75% 

Box  3 

Development 

33% 
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Box  3 

ORS  office  proposes  existing  capability 

33% 

Box  3 

ORS  proposed  solutions  rejected 

33% 

V.  Air  Force  ACAT  //////  Acquisition 

Differences  between  ACAT  I  and  ACAT  II /III 

The  major  discriminator  between  managing  an  ACAT  I  program  and  ACAT  ll/lll 

programs  is  the  rank/position  of  the  Milestone  Decision  Authority  (MDA).  The  MDA  is 
the  designated  individual  with  overall  responsibility  for  a  program  that  has  the  authority 
to  approve  entry  of  a  program  into  the  next  phase  of  the  acquisition 
process(USD(AT&L),  2007).  For  almost  all  Air  Force  space  programs,  the  SMC 
Commander  is  appointed  as  the  PEO  but  MDA  authority  is  only  delegated  to  them  for 
ACAT  ll/lll  programs.  The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics  (USD(AT&L)  retains  MDA  authority  over  all  ACAT  I  programs.  This  essentially 
means  that  all  major  decisions  and  documents  for  ACAT  I  programs  must  pass  through 
another  level  of  review,  creating  more  steps  in  the  process  and  obviously  taking  more 
time.  This  fact  was  the  primary  driver  for  most  modifications  to  ERAM  2.1  to  create 
ERAM  2.2. 

The  difference  in  cost  of  the  different  ACAT  levels  also  drives  a  major  difference 
in  how  programs  are  now  managed.  In  the  past,  space  ACAT  I  programs  were  given  all 
the  attention  because  as  one  HQ/AF  expert  noted,  "we  try  to  make  every  satellite  into  a 
Battlestar  Galactica."  He  then  went  on  to  explain  that  high  launch  costs  cause  planners 
to  try  to  get  as  much  capability  on-orbit  with  each  satellite.  Essentially,  since  a  launch 
was  a  fixed  cost,  the  expectation  was  to  maximize  the  capability  per  launch.  When  this 
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concept  was  presented  to  an  AFSPC  capability  planner  a  during  research  interview,  she 


emphasized  that  this  paradigm  has  primarily  ended.  Due  to  recent  and  looming  funding 
cutbacks,  AFSPC  is  now  focused  on  "keeping  requirements  at  a  basic  level  and  then 
performing  block  upgrades."  She  noted  that  they  are  trying  to  keep  programs  at  the 
ACAT  II  or  lower  level  to  ensure  funding  is  available.  This  paradigm  shift  shaped  the 
results  for  ERAM  2.2  because  there  are  multiple  steps  within  the  process  where  funding 
needs  to  be  procured.  SMEs  noted  that  the  timelines  to  gather  funds  for  smaller 
programs  was  much  shorter. 

Another  of  the  major  differences  between  ACAT  I  programs  and  ACAT  ll/lll 
programs  is  the  reporting  requirement.  US  Code  Sec  2432  requires  that  all  Major 
Defense  Acquisition  Programs  (MDAP)  submit  status  reports  on  the  program  for 
Congressional  review.  These  reports  are  known  as  Selected  Acquisition  Reports  (SAR) 
and  are  required  to  be  submitted  quarterly.  They  summarize  the  latest  estimates  of 
cost,  schedule  and  performance  status  for  the  program  and  identify  deviations  from 
past  reports.  The  requirements  to  be  considered  an  MDAP  are  more  stringent  than 
ACAT  I  programs  but  essentially  are  the  same.  The  current  SMC  ACAT  I  portfolio 
consists  of  17  programs  and  14  of  them  are  currently  considered  MDAPs  but  no  ACAT 
ll/lll  programs  are  MDAPs  (USD  (AT&L),  2010). 

The  fact  that  SARs  are  optional  but  not  required  for  ACAT  ll/lll  programs 
presented  a  major  problem  in  gathering  quantitative  data  for  this  research  objective. 
When  asked  about  this  requirement,  one  program  manager  noted;  "SARs  are  optional 
but  our  PEO  doesn't  want  them  so  we  don't  do  them."  None  of  the  five  program 
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managers  interviewed  had  ever  seen  a  SAR  completed  for  anything  but  an  ACAT  I 
program.  In  addition  to  SARs,  Monthly  Acquisition  Reports  are  required  for  all  Air  Force 
acquisition  programs.  For  unclassified  programs,  these  reports  are  uploaded  to  a 
database  called  System  Metric  and  Reporting  Tool  (SMART).  SMART  is  a  collaborative 
web-based  application  designed  to  assist  program  managers  and  acquisition  leaders 
with  reporting  and  viewing  program  health  and  status(Defense  Acquisition  University, 
2009).  While  SMART  has  a  wealth  of  data,  it  only  has  data  back  to  2010.  In  addition, 
ACAT  ll/lll  programs  were  only  required  to  file  MARs  in  SMART  starting  in  April  2012. 

With  no  central  repository  for  ACAT  ll/lll  data,  interviews  and  documents 
provided  by  the  SMEs  had  to  serve  as  the  primary  sources  of  data  for  modifying  ERAM 
2.1  to  ERAM  2.2.  In  many  cases,  the  respondents  could  not  provide  exact  data  but 
instead  provided  "rules  of  thumb"  based  on  their  experiences.  Meyer  and  Booker 
identify  this  type  of  data  gathering  as  very  useful  when  trying  "to  provide  estimates  on 
new,  rare,  complex,  or  otherwise  poorly  understood  phenomena"  (Meyer,  2001).  The 
goals  of  this  research  fit  these  criteria  very  well.  The  positive  aspects  of  this  type  of  data 
for  the  model  is  that  it  is  based  upon  experts  in  their  field  that  have  years  of  experience 
in  their  positions.  The  negative  aspects  are  that  it  is  possible  that  "some  people 
"rewrite"  the  past  to  make  it  more  consistent  with  current  beliefs  or  remember  the  past 
in  a  self-enhancing  way"  (Neuman,  2006).  With  these  cautions  and  caveats  noted,  the 
following  analysis  was  done. 


51 


Analysis  of  ACAT  II /III  Programs 

After  consulting  with  experts  at  SMC,  AFSPC  and  Headquarters  Air  Force,  ERAM 

2.1  was  reviewed  based  on  their  feedback.  Points  in  the  model  were  identified  that 
needed  to  be  modified  to  accommodate  variations  in  ACAT  level.  Changes  to  the 
model  were  dependent  on  four  factors. 

1.  MDA  for  ACAT  I  programs  is  USD(AT&L)  which  drives  extra  preparation 
and  reviews  for  these  programs.  MDA  for  ACAT  ll/lll  programs  is  SMC/CC 
so  preparation  time  and  the  number  of  reviews  are  reduced. 

2.  ACAT  ll/lll  programs  are  less  costly  so  contracts  and  documents  are 
usually  shorter. 

3.  ACAT  ll/lll  programs  are  less  costly  so  the  time  to  gather  funds  is  reduced 
in  most  cases. 

4.  Testing  duration  is  normally  reduced  for  ACAT  ll/lll  programs  since  the 
program  is  not  as  complicated.  ACAT  I  programs  usually  have  many  more 
requirements  and,  therefore,  more  test  objectives. 

These  factors  were  based  on  policy  documents  such  as  DoDI  5000.02  and  the  SMC 
Acquisition  Development  Strategy  Guide  as  well  as  the  feedback  provided  by  SMEs  at 
SMC,  AFSPC  headquarters  and  SAF/AQ.  The  following  sections  detail  the  steps  in  the 
ERAM  model  that  were  identified  as  needing  medication  based  on  ACAT  level. 
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Prior  to  Milestone  A:in  ERAM 


Table  9:  Modified  Duration  Data  for  ERAM  Steps  Prior  to  MS-A 


Distributions  bv  ACAT  Level 

1 

II 

III 

Step  Name 

Min 

Max 

Most 

Likely 

Min 

Max 

Most 

Likely 

Min 

Max 

Most 

Likely 

Prepare  for 
Acquisition  Panels 

40 

60 

56 

15 

60 

35 

15 

45 

25 

Check  for  Funding 

30 

270 

120 

30 

360 

60 

15 

360 

60 

Draft  RFP  Prep 

20 

40 

34 

10 

20 

17 

10 

20 

17 

RFP  Coordination 

Process 

50 

1200 

370 

55 

297 

165 

55 

297 

165 

Prep  w/  PEO 

15 

35 

30 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

The  differences  in  each  step  are  explained  below: 


Prepare  for  Acquisition  Panels  -  Program  managers  noted  that  preparation  time  was 
weeks  longer  for  ACAT  I  programs.  More  organizations  were  required  to  review  all 
documents  and  they  each  provided  feedback  in  preparation  for  the  acquisition  panels. 


Check  for  Funding  -  Many  times  a  program  will  need  to  compete  for  funding  against 
other  needs.  Due  to  budget  cuts,  it  is  easier  to  procure  funding  for  smaller  programs 
now. 


Draft  RFP  Prep  -  Data  provided  by  SMC  acquisition  officials  showed  that  Request  for 
Proposal  (RFP)  for  ACAT  I  programs  took  twice  as  long  as  smaller  programs.  They  may 
be  much  longer  documents  or  have  more  requirements. 
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RFP  Coordination  Process  -  More  organizations  want  to  see  the  RFP  for  ACAT  I  programs 


than  ACAT  ll/lll  programs  driving  a  longer  timeline 


Prepare  with  PEO  -  ACAT  I  programs  require  additional  meetings  with  the  PEO  and  staff 
to  prepare  for  MDA  meetings. 
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Prior  to  Milestone  B  in  ERAM 


Table  10:  Modified  Duration  Data  for  ERAM  Steps  Prior  to  MS-B 


Distributions  bv  ACAT  Level 

1 

II 

III 

Name 

Min 

Max 

Most 

Likely 

Min 

Max 

Most 

Likely 

Min 

Max 

Most 

Likely 

Contract  Length 

365 

2190 

1980 

365 

2190 

1095 

365 

2190 

550 

Prepare  for 
Acquisition  Panels 

40 

60 

56 

15 

60 

35 

15 

45 

25 

Prep  w/  PEO 

15 

35 

30 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Independent  cost 
estimate 

30 

60 

35 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Acquisition  Planning 

60 

1200 

300 

60 

330 

210 

60 

330 

210 

Testing  length(TL)  for  ACAT  1  is  25%  of  Contract  Length. 

Developmental 

Testing 

Testing  length(TL)  for  ACAT  ll/lll  is  15%  of  Contract  Length. 
Min=75%  of  testing  length 

Max==l.l%  of  testing  length 

Most  Likely=testing  length 

Funding  required 

90 

270 

225 

30 

270 

90 

30 

270 

90 

Start  of  Test  & 

If  ACAT  1,  has  75%  of  contract  elapsed,  otherwise  if  ACAT  ll/lll 

Evaluation 

has  85%  of  contract  elapsed. 

PEM/Other  Staff 
find  money 

14 

270 

83 

5 

270 

160 

5 

270 

160 

The  differences  in  each  step  are  explained  below: 

Contract  Length  -  ACAT  I  programs  normally  have  a  much  longer  contract  due  to  the 
size,  cost  and  complexity  of  these  programs 
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Prepare  for  Acquisition  Panels  -  Program  managers  noted  that  preparation  time  was 


weeks  longer  for  ACAT  I  programs.  More  organizations  were  required  to  review  all 
documents  and  they  each  provided  feedback  in  preparation  for  the  acquisition  panels. 

Prepare  with  PEO  -  ACAT  I  programs  require  additional  meetings  with  the  PEO  and  staff 
to  prepare  for  MDA  meetings. 

Independent  Cost  Estimate  -The  Weapons  System  Acquisition  Reform  Act  of  2009 
requires  independent  cost  estimates  for  all  MDAP  programs,  and  therefore  almost  all 
ACAT  I  programs.  Air  Force  Cost  Analysis  Agency  SMEs  noted  that  independent  cost 
estimates  are  optional  but  never  completed  for  ACAT  ll/lll  programs. 

Acquisition  Planning  -  Data  provided  by  SMC  acquisition  officials  showed  that  acquisition 
strategy  planning  follows  the  same  process  regardless  of  ACAT  level  but  usually  takes 
longer  for  ACAT  I  programs. 

Developmental  Testing  -  Program  managers  noted  that  the  amount  of  testing  is  usually 
based  on  contract  length.  The  way  this  data  is  derived  is  that  ERAM  will  first  pick  a 
contract  length  using  the  Monte  Carlo  simulation.  It  will  then  calculate  a  testing  length 
(TL)  and  then  scaling  factors  are  applied  to  create  a  triangular  distribution  based  on  this 
TL. 

Funding  required  -  Many  times  a  program  will  need  to  compete  for  funding  against 
other  needs.  Due  to  budget  cuts,  it  is  easier  to  procure  funding  for  smaller  programs 
now. 
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Start  of  Test  and  Evaluation(T&E)  -  ERAM  2.1  was  built  to  not  start  T&E  until  a  certain 


percentage  of  the  acquisition  contract  had  elapsed.  SMEs  identified  that  this  percentage 
is  slightly  different  for  ACAT  ll/lll  programs. 

PEM/Other  Staff  find  money  -  Many  times  a  program  will  need  to  compete  for  funding 
against  other  needs.  Due  to  budget  cuts,  it  is  easier  to  procure  funding  for  smaller 
programs  now  at  this  point  in  an  acquisition  program,  ACAT  I  programs  are  usually  "too 
big  to  fail"  so  funding  is  easier  to  acquire. 
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Prior  to  Milestone  C  in  ERAM 


Table  11:  Modified  Duration  Data  for  ERAM  Steps  Prior  to  MS-C 


Name 

Dis 

1 

...  Most 

Min  Max  , ..  . 

Likely 

tributions  bv  ACAT  Le 

II 

...  ..  Most 

Min  Max  ...  . 

Likely 

vel 

III 

...  Most 

Min  Max  , , 
Likely 

Contract  Length 

365  2190  1980 

365  2190  1095 

365  2190  550 

Integrated  Testing 

Testing  length(TL)  for  ACAT  1  is  25%  of  Contract  Length. 

Testing  length(TL)  for  ACAT  ll/lll  is  15%  of  Contract  Length. 

Min  =  75%  of  TL 

Max  =  1.1%  of  TL 

Most  Likely  =  TL 

Develop  System  & 
Live  Fire  Test  & 
Operational 
Readiness  Test 

Testing  length(TL)  for  ACAT  1  is  7%  of  Contract  Length. 

Testing  length(TL)  for  ACAT  ll/lll  is  5%  of  Contract  Length. 

Min  =  75%  of  TL 

Max  =  1.1%  of  TL 

Most  Likely  =  TL 

Acquisition  Planning 

60  1200  300 

60  330  210 

60  330  210 

Funding  Required 

90  270  225 

30  270  90 

30  270  90 

Prepare  for 
Acquisition  Panels 

40  60  56 

15  60  35 

15  45  25 

Prep  w/  PEO 

15  35  30 

N/A  N/A  N/A 

N/A  N/A  N/A 

Program  Review 

25  35  30 

25  35  30 

25  35  30 

PEM/Other  Staff 
find  Money 

14  270  83 

5  270  50 

5  270  30 

The  differences  in  each  step  are  explained  below 

Contract  Length  -  ACAT  I  programs  normally  have  a  much  longer  contract  due  to  the 
size,  cost  and  complexity  of  these  programs 
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Integrated  Testing  -  As  noted  under  Developmental  Testing  in  the  previous  section. 


program  managers  indicated  that  the  amount  of  testing  is  usually  based  on  contract 
length.  The  way  this  data  is  derived  is  that  ERAM  will  first  pick  a  contract  length  using  a 
Monte  Carlo  simulation.  It  will  then  calculate  a  testing  length  (TL)  and  then  scaling 
factors  are  applied  to  create  a  triangular  distribution  based  on  this  TL.  This  case  is  for 
combined  developmental  and  operational  testing. 

Develop  System  &  Live  Fire  Test  and  Operational  Readiness  Test  -  If  a  separate  live  fire 
and  operational  test  is  conducted  it  is  also  usually  a  percentage  of  contract  length  but 
slightly  shorter  for  ACAT  ll/lll  program.  See  explanation  under  Integrated  testing. 

Acquisition  Planning  -  Data  provided  by  SMC  acquisition  officials  showed  that  acquisition 
strategy  planning  follows  the  same  process  regardless  of  ACAT  level  but  usually  takes 
longer  for  ACAT  I  programs. 

Funding  required  -  Many  times  a  program  will  need  to  compete  for  funding  against 
other  needs.  Due  to  budget  cuts,  it  is  easier  to  procure  funding  for  smaller  programs 
now. 

Prepare  for  Acquisition  Panels  -  Program  managers  noted  that  preparation  time  was 
weeks  longer  for  ACAT  I  programs.  More  organizations  were  required  to  review  all 
documents  and  they  each  provided  feedback  in  preparation  for  the  acquisition  panels. 

Prepare  with  PEO  -  ACAT  I  programs  require  additional  meetings  with  the  PEO  and  staff 
to  prepare  for  MDA  meetings. 
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PEM/Other  Staff  find  money  -  Many  times  a  program  will  need  to  compete  for  funding 
against  other  needs.  Due  to  budget  cuts,  it  is  easier  to  procure  funding  for  smaller 
programs  now  at  this  point  in  an  acquisition  program,  ACAT  I  programs  are  usually  "too 
big  to  fail"  so  funding  is  easier  to  acquire. 
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VI.  Conclusions 


Tools  such  as  ERAM  2.2  are  geared  to  help  acquisition  planners  improve  their 
skills  and  provide  early  schedule  estimates  based  upon  existing  statutes,  policy  and  most 
importantly  historical  data.  Improvements  in  this  area  are  necessary  because  the 
"average  total  cost  growth  factor  for  completed  Major  Defense  Acquisition  Programs 
(MDAPs)  was  46  percent"  (Younossi,  2008).  While  this  data  includes  programs  from  all 
services,  this  research  has  shown  that  space  programs  are  not  any  different. 

ERAM  2.2  can  help  SMC  acquistion  planners  better  map  out  programs  in  the 
intial  stages  of  development.  While  they  previosuly  only  had  the  ability  to  look  at  ACAT 
I  programs,  this  version  now  allows  the  model  to  be  fine-tuned  for  all  ACAT  levels  as 
well  as  different  types  of  rapid  programs. 

While  tools  such  as  ERAM  are  useful,  it  is  obvious  that  it  will  not  affect  programs 
unless  the  lessons  learned  are  applied  to  acquistion  programs.  A  recent  article  points  to 
a  "conspiracy  of  optimism"  where  military  planners  want  to  obtain  a  capability  they 
cannot  afford,  or  which  is  at  the  risky  end  of  the  spectrum.  To  avoid  scaring  those 
assessing  or  paying  the  bills,  they  either  under-estimate  how  much  it  will  cost,  or  they 
fudge  the  risk  of  the  project,  or  both  (Sweetman,  2012).  If  this  type  of  behavior  is  true, 
then  it  does  not  matter  how  many  good  tools  are  developed  because  their 
recommendations  will  be  ignored. 
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Areas  of  Further  Research 

ERAM  2.2  is  an  improvement  to  ERAM  2.1  but  it  is  definitely  not  a  complete 
product.  This  research  utilized  analysis  of  existing  policy  and  guidance,  interviews  with 
SMEs  and  acquisition  program  documentation  provided  by  the  SMEs.  Even  though 
ERAM  2.2  will  be  useful  for  SMC  planners,  listed  below  are  additional  topics  that  can  be 
explored  to  improve  the  model  or  utilize  it  in  different  ways. 

1 .  Air  Staff  acquisition  policy  SMEs  noted  that  Congress  signs  into  law  over 
1000  changes  that  affect  the  DoD  annually.  It  usually  takes  2-3  years  for  the 
appropriate  documents  to  be  modified  and  distributed  to  program  managers 
and  planners.  Due  to  these  constant  changes,  ERAM  can  never  remain  a  static 
tool  if  it  is  to  remain  useful.  Annual  maintenance  will  be  required  to  keep  it 
up  to  date  with  new  policy  and  to  also  ensure  that  representative  data  sets  are 
current. 

2.  Leach  and  Searle  pointed  out  the  ERAM  could  be  the  “backbone  of  a  training 
program”(Leach  &  Searle,  2011).  SMC/XR  recommended  to  Defense 
Acquisition  University  that  they  learn  more  about  ERAM  with  the  intent  that 
they  can  use  it  to  train  DoD  and  industry  PMs. 

3.  ERAM  could  be  tailored  to  any  kind  of  acquisition  program,  not  just  space. 
Other  types  of  acquisition  programs  such  as  information  systems,  aircraft  or 
naval  systems  could  benefit  from  a  version  of  ERAM  modified  to  fit  their 
process. 

4.  Many  parts  of  this  model  are  still  incomplete  or  based  on  a  small  data  set. 
Further  research  is  definitely  needed  to  fill  in  the  gaps  that  were  found  in 
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when  looking  at  the  work  of  organizations  such  as  AF  TENCAP,  ORS  Office 
and  AF  RCO. 


Appendix  I  -Rapid  Acquisition  Program  Raw  Data 


Table  12:  JUON  Data 


Program  1 

Program  2 

Program  3 

Warfighter 

submits 

requirement 

Validation 

MDD 

Initial  Fielding 
Assess 

Suitability 

Adjust  solutions 

6-Jun-10 

Days 

12-Aug-ll 

Days 

12-Oct-07 

Days 

17-Sep-10 

103 

6-Sep-ll 

25 

28-Oct-07 

16 

24-Feb-12 

13-Dec-10 

87 

31-Aug-12 

360 

13-Feb-ll 

62 

13-Nov-ll 

273 

Final  disposition 

TBD 

TBD 

Rejected 

JRAC 

Program  4 

Program  5 

Program  6 

Warfighter 

submits 

requirement 

Validation 

MDD 

Initial  Fielding 
Assess 

Suitability 

Adjust  solutions 

8-Jun-06 

Days 

25-Aug-06 

Days 

28-Jun-06 

Days 

30-Jun-06 

22 

13-Sep-06 

19 

27-Oct-06 

121 

l-Mar-07 

125 

Final  disposition 

Rejected 

JRAC 

Rejected 

JRAC 

Sustain 

existing 

Program  7 

Program  8 

Program  9 

Warfighter 

submits 

requirement 

Validation 

MDD 

2-Jun-ll 

Days 

10-May- 

07 

Days 

15-Jul-09 

Days 

30-Jun-ll 

28 

12-Jun-07 

33 

16-Jul-09 

1 

ll-Sep-07 

63 


Initial  Fielding 
Assess 

Suitability 

Adjust  solutions 

17-Oct-ll 

109 

28-Oct-ll 

11 

7-Nov-ll 

10 

Final  disposition 

Demilitarize/dispose 

Transition  enduring 

Rescinded  10  Jul 

10 

Table  13:  TENCAP  Data 


TC  #1 

TC  #2 

TC  #3 

Requirement 

4/25/2011 

Days 

2/5/2010 

Days 

1/1/2010 

Days 

Development  start 

7/5/2011 

71 

2/15/2010 

10 

1/2/2010 

1 

Development  complete 

2/10/2012 

220 

12/15/2010 

303 

11/15/2010 

317 

Transition  to  user 

2/15/2012 

5 

2/15/2011 

62 

1/10/2011 

56 

Project  complete 

2/16/2012 

1 

10/15/2011 

242 

5/15/2011 

125 

Requirement 
Development  start 
Development  complete 

Transition  to  user 

Project  complete 

TC  #4 

TC  #5 

TC  #6 

TC  #7 

1/1/2011 

Days 

12/1/2009 

Days 

8/16/2011 

Days 

3/3/2010 

Days 

2/15/2011 

45 

3/20/2010 

109 

8/20/2011 

4 

3/15/2010 

12 

10/1/2011 

228 

8/20/2010 

153 

6/15/2012 

300 

4/29/2010 

45 

10/3/2011 

2 

9/1/2010 

12 

6/22/2012 

7 

5/16/2010 

17 

11/11/2011 

39 

10/14/2010 

43 

6/29/2012 

7 

7/15/2010 

60 

Table  14:  ORS  Data 


ORS-1 

Requirement 

3/1/2008 

Days 

STRATCOM  approval 

5/1/2008 

61 

Executive  committee  approval 

10/1/2008 

153 

Solution  chosen 

6/1/2009 

243 

Deployment 

6/1/2011 

730 
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Appendix  II  -  Sample  Interview  Questions 


1 .  Describe  your  roles  and  responsibilities  in  the  space  acquisition  processes. 

2.  What  space  programs  do  you  have  experience  with? 

3.  Describe  what  processes  you’ve  been  involved  with  regards  to  space  capability 
development  (i.e.  gap  analysis,  S&T,  JCIDS  documents,  DP,  and/or  acquisitions)  and 
with  which  program.  What  level  was  the  program? 

4.  Describe  the  specific  activities  and  decision  points  for  the  processes  you’ve  been 
involved  in  from  question  3. 

5.  What  do  you  see  as  the  primary  differences  between  AC  AT  I/II/III  programs?  Were 
there  specific  activities  that  were  different? 

6.  What  reviews  and  documentation  were  required  for  your  program? 

7.  Describe  the  role,  if  any,  you  had  with  rapid  programs. 

8.  What  was  the  timeframe  for  approval  on  the  required  documentation?  Which 
organizations  were  required  for  coordination?  What  obstacles  needed  to  be  overcome? 

9.  Have  you  experienced  delays  with  your  programs?  If  so,  what  would  you  say  was  the 
primary  driver  of  that  delay? 

10.  When  did  the  various  reviews  and  other  meetings  occur  in  your  program?  Who  was 
required  to  attend?  What  were  the  challenges  that  arose?  How  did  you  overcome  those 
challenges? 

1 1 .  If  applicable,  what  waivers  were  applicable  for  moving  your  program  forward?  How  did 
you  obtain  approval  for  those  waivers? 

12.  Would  you  be  willing  to  provide  data  on  the  durations  of  the  activities  and  probabilities 
for  decision  points? 

13.  Is  there  anyone  specific  that  you  recommend  I  talk  to  about  these  topics? 
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